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SUMMARY 


The Critical Tracking Teat (CTT) la a paychomotor teat that haa 
proven In the paat to allow reliable aeaaurement of human operator per- 
formance, and haa been shown to give aenaltlve decrementa to a variety 
of atreaaea. Currently the CTT la being tested aa a method of detecting 
human operator Impairment. A paaa level la aet for each subject, baaed 
on *.>iat subject's asymptotic skill level %ihlle sober. It la critical 
that complete training take place before the Individualized pass level 
la set In order that the Impairment can be detected. 

Some subjects have shown erratic learning trends typical of unmoti- 
vated performance. These subjects seemed more motivated to finish their 
training sessions quickly, rather than to receive monetary rewards for 
good performance. 

This paper describee eperant conditioning procedurss, specifically 
the use of negative reinforcement. In achieving stable learning beha- 
vior. These results now provide a more general basis for the applica- 
tion of reward /penalty structures In manual control research- 


INTRODUCTION 


The Critical Tracking Test (CTT) is ». psychomotor task based on man- 
ual control principles (Reference I) that has been highly developed and 
applied to a variety of research problems (Reference 2)« CTT perfor- 
mance la affected by individual differences and abilities, and critical 
Instability scores (X^) vary across subjects. For a given trained sub- 
ject, however, Is quite consistent, and Is sensitive to blood alcohol 
concentration. Thus, It Is possible to set a pass /fall level required 
for a particular decision strategy (Reference 2). It Is Irportant that 
the pasa criterion accurately reflect the subject's skill level. If 
^pass operator will be able to pass while Intoxicated. 
It Xpagg Is too high, the sober operator will experience unnecessary and 
annoying delays while the equipment resets. One objective of this 
experiment was the development and refinement of a rational a priori 
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procedure for setting pass levels based on training scores only. Be- 
cause of motivational problems encountered In the original Experiment, a 
subsequent Training Experiment was carried out. 


r' 


EXPERIMENT 


Background 

For the experiment, 24 subjects were obtained through the Los 
Angeles Municipal Courts, who had plead guilty to charges of Driving 
Under the Influence of Alcohol (DUX). The judge offered them the option 
of participating In this research project Instead of the usual $350 
traffic fine and traffic school. Even Chough Che monetary rewards for 
participating were substantial. It was still a choice between Che lesser 
of two evils. Subject participation was often reluctant, and In some 
cases subjects were effectively participating under duress. 

The MMPI was administered and subjects with clinically abnormal pro- 
files were eliminated from the population. The subjects were trained Co 
"drive" In a simulator and to perform the critical tracking task. Three 
sessions of approximately 2 hours each on separate days were required 
for training. Following training, subjects participated In 3 experi- 
mental alcohol sessions, scheduled , about a week apart, and lasting 10 to 
12 hours. Each subject also participated In a follow-up session of 
approximately 1/2 hour. They received $3. 10/hour for training and ex- 
perimental sessions, and a bonus schedule was worked out Co provide 
Incentives for good performance and completion of the program (e.g.. 
Reference 2). The subjects averaged about $220 total from wages and 
bonus money and the court cancelled a $350 traffic fine If they satis- 
factorily completed all requirements. 

Learning curves were generated from training data to determine If 
asymptotic levels of performance had been attained. The 24 training 
plots were evaluated for Indications of poor motivation or Inadequate 
training. Half of the subjects showed lack of motivation and/or not 
enough training. These curves were characterized by unusually low 
scores In Che last training session and a slope that was increasing even 
at the end of Che last session (further discussion of this problem Is 
found In section entitled Results). The remaining subjects showed 
asymptotic learning and consistent behavior, l.e.. Figure 1. The simple 
pass criteria chosen for these 12 subjects correlated 0.862 with the 
"ideal" or hindsight criteria taken from the later experimental data as 
shown In Figure 2. 


Methods 

CTT training was done In 3 sessions on separate days. Each subject 
read the Instructions (Appendix) and was Informed of Che bonus structure 
(at least 1 pass in a block of 4 trials > 75c)« Each training session 
consisted of 25 blocks of 4 trials, or 100 trials. 
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Figure 1. Good Asymptotic Training Exhibited 
by One Test Subject 
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Statistically Determined Perfect Pass Criterion 


Figure 2. Pass Criterion Analysis 


The Initial pass criterion was set at an arbitrary low level that 
all subjects could pass within a few trials (l.e., X * 2.9). If the 
subject passed 3 out of 4 trials In a block, the pass level was raised 
0.1. If the subject passed all 4 trials the pass level was raised 0.2. 
If the subject failed all 4 trials the pass level was lowered 0.1. The 
pass level could go back to, but not below, a level where 3 out of 4 
trials had previously been passed. This strategy tends to prevent deli- 
berate "backsliding” by the subjects. 

Breaks were given every 10 or 15 minutes. The mean X,. for each 
block In the last training session was computed. The experimental 
Xp was chosen by taking the median of the highest 3 block means and 
reducing this value by 0.3 (roughly 1/2 of <jx). This procedure was 
found to give a pass level consistent with a one trial probability of 
failure of 40 percent which was required for a one-pass-ln-four-atterapts 
decision strategy as discussed In Reference 2. 


Results 

For this experiment we were Interested in determining the extent to 
which we were able to predict failure rates of Impaired operators based 
on sober training data, and thus needed to predict an accurate pass 
score (Xp^gg). The experimental (alcohol) session data (all sober base- 
lines and placebo trials) were analyzed to determine In hindsight what 
the "Ideal" (a posteriori) pass/fall score for each subject should have 
been to Insure a 40 percent probability of falling a single trial when 
sober (as discussed In Reference 2). Comparing the experimental criter- 
ion set during this Ideal criterion we found that In 13 cases the 
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^pass (Figure 2). The solid circles indicate those subjects 
who were well trained. For these individuals our predicted pass level 
was remarkably consistent with the statistically determined perfect pass 
level. Problems were found when we compared predicted and perfect pass 
levels for subjects with both minor and major training problems (i*e., 
erratic learning curves). Subjects with minor problems exhibited a 
remarV.able and consistent increase in scores as soon as there was no 
time benefit to failing. Subjects with major training problems had dis- 
played problems consistently during training, and these problems persis- 
ted in the experiments* Even though the Individual pass criteria were 
not perfect, the discrirainabillty results agree with the statistical 
model described in the companion paper (Reference 2). 


Discussion 

Because 15 of the 24 subjects were assigned Xp^ss criteria that were 
too low, the incentive structure was re-evaluated. The 75c bonus had 
been given for passing at least one trial in a block of 4. (This was 
chosen because it simulates the one pass out of four attempts strategy 
used in the formal validation experiment trials in the reanalysls of 
prior data (Reference 2)* We found that most of the subjects did not 

even bother to learn the reward structure In order to maximize the 
bonuses earned (the bonus money was added across sessions and paid out 
at the completion of the experiment). They preferred to put forth a 
random effort and accept whatever total monies they happened to earn. 

Research in operant conditioning (l.e., Reference 4) has shown that 
the more delayed such a *'reinf orcer” is, the less potent it is. There- 
fore, while the bonus money may have reinforced the subject for complet- 
ing the experiment. It had little effect on each Individual trial of the 
CTT. It was also observed during training that the subjects' primary 
motivation was to complete training sessions as soon as possible, even 
to the extent of foregoing rest breaks and falling the test quickly to 
speed up the trial repetition rate. 

It was decided that in order to elicit a consistent and stable per- 
formance from the subject a more immediate relnforcer should be applied 
after each passed trial on the CTT. Reinforcement occurs when a reward- 
ing stimulus follows a response or when a negative or punishing stimulus 
is avoided. It was decided that, since the training procedure is so 
tedious, a negative, or punishing, stimulus would be a 30 second "time 
out" condition added after each failed trial. By using the avoidance 
paradigm, the absence of the aversive stimulus (the "time out") becomes 
a relnforcer (Reference 4). 

A subsequent study was conducted to verify if addition of this "time 
out" procedure would motivate the subjects in order that stable and com- 
plete training is obtained in three sessions. 
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TRAINING EXPERIMENT 


Objectives 

As described earlier, each subject Is required to complete 300 
trills on the CTT to establish an Individualized pass/fall score. This 
is a tedious process done over 3 sessions, each lasting 2 to 3 hours. 

Becauac of the poor performance and comments made during the experi- 
ment, v«e conducted a brief training experiment using the reinforcement 
strategy described above. In this situation a green "pass" light In the 
display of the CTT apparatus takes on new reinforcing properties, as It 
now signals the absence of a "time out." In the previous experiment the 
green light. In general, only provided Information as to the outcome of 
the trial. The red "fall" light, on the other hand, now takes on aver- 
sive properties because it Is present during the 30 second "time out." 
The red light was also merely Informational In the prevlo'^ experiment. 


Procedure 

Six additional subjects were contacted and enrolled through the Los 
Angeles Courts, as before. One hundred dollars of their $350 fine was 
dropped when they completed the project. The MMPI was administered as 
In the earlier Experiment. The first two subjects were given the 
printed Instructions (Appendix) that outlined the objectives of the 
study and the bonus structure. They repeatedly expressed their surprise 
at being paid, because traffic schools do not pay for participation. 
Also, as In the earlier experiment, they chose not to learn the Incen- 
tive structure and said, In effect, "You just keep track and pay me 

later." It was decided at that point that paying the subjects was 

superfluous and unnecessary, so the next four subjects received verbal 
instructions with no mention of wages or incentives. 

The subjects received the same training procedure as In the earlier 

experiment (25 blocks of 4 trials each), except that each time a trial 

was failed there was a 30 second delay. [Note: These subjects all came 

after work and, because of the hour and the time of year, the testing, 
which took place In a parked car, was done In the dark. When the trial 
was completed, the display light went off with only the red "FAIL" or 
green "PASS" light remaining on, depending on the outcome of the trial. 
This meant that for a failed trial the subject waited In a dark car for 
30 seconds, looking at a small red light that said "FAIL."] 


Results 

Some of the learning curves are shown In Figures 3->5 for the new 
condition using the "time out" procedure. There Is no qualitative dif- 
ference between those subjects that were paid and those that were not 
paid. The learning curves show the same asymptotic learning curves that 
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Figure 3. Learning Curve for a Subject Who Received 
Delayed Monetary Rewards for Task Performance in 
Addition to Iwnedlate Walt Time Penalties 
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yielded accurate criteria predictions in the earlier experiment. Fur- 
thermore, the t rial-to-trial consistency is better, as shown by the low 
standard deviations at the bottom of Figures 3-5. 


CONCLUDING REMARKS 


This study has shown that, to make an accurate prediction of a pass 
criterion, stable and complete learning must take place. It has also 

been shown that learning is facilitated when the subject is reinforced 

after each trial. In the past, subjects were given numerical feedback 
on each CTT trial as to what their actual scores were (i.e., References 
5-7). This acted as an immediate and powerful reinforcer, because each 
trial provided an opportunity to better their last score. In addition, 

we were fortunate in working only with motivated subjects. For these 

reasons our earlier Reward/Penalty structures (Reference 3) were based 
on a positive reinforcement model only. In this experiment the subject 
was trained to "pass the test" without being aware that the eventual 
pass level was adjusted to their individual ability to perform. Since 
the normal reinforcement by display of (TIT scores was not used, the 30 
second time out after each fail was substituted. 

In assessing what reinforcers are available for use with unmotivated 
subjects, money seems to be a neutral stimulus. The main motivator 
should be based on passing the test to avoid the aversive stimulus. In 
this way subjects will learn to pass the test without being aware that 
the eventual pass level is adjusted according to their individual abil- 
ity to perform. The "30 second time out for fail" procedure seems to 
provide the desired immediate reinforcement after each trial and main- 
tains consistent test scores. In addition, these results allow for a 
more general application of reward/penalty structures in manual control 
research depending on the basic motivation for subject participation. 
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APPENDIX 


This study Is being conducted for the U.S. Department of Transporta- 
tion to test a device designed to tell a driver that he may be too drunk 
to drive. Your participation In this project Is Important because you 
will be contributing to the Improvement of measures for reducing alcohol 
related accidents. This phase of the study Is aimed at helping us fin- 
alize our training strategy for the task. 

You will use the device to perform a steering control task. We will 
provide you with an opportunity to become familiar with the operation of 
the device during three separate sessions. Each session will last 
approximately three hours. 

The steering control task will be performed on a unit which Is 
mounted on the side of the steering wheel In a specially modified car. 
The device consists of a meter (a display and a needle), a start switch, 
and a small computer In the trunk of the car. When the task begins, the 
needle Is centered In the green area of the display. As the task pro- 
ceeds, the needle begins to wander either to the left or the right. It 
Is your job to keep the needle centered in the green area by moving the 
steering wheel in the direction you want the needle to move. It will 
become Increasingly difficult to keep the needle centered. 

The device will automatically set a passing level for your perfor- 
mance. This pass level is based on learning rates established in pre- 
vious research. Trials will be conducted in groups of four. If you 
pass one trial in a group of four, you will receive a cash reward. If 
you have already passed one trial, it is important to continue to do 
your best on the remaining trials, since the task will continue to 
increase in difficulty, and your ability to pass future trials will be 
affected. The level of difficulty will change after each set of four 
trials. It will be necessay for the experimenter to reset the system 
periodically. 
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We expect that you will do poorly on the first few trials because 
you are completely unfamiliar with the task. You should not become dis- 
couraged; we know that your performance will Improve with training. 

You will be paid $3/hour plus a bonus of 75 cents per group passed 
for each session. You can expect to average $12-14 per session/ 

Your pay will be divided up Into 2 parts — each day you will take 
home your hourly wages, thus on an average day you take home about 
$9.00. Your bonus money Is held In a holding account, to be paid to you 
on your last day of experiments (approximately $13.50 In addition to 
your hourly pay). 

Should you drop out sometime during the experiment, your holding 
account money will still be owed to you, but It can't be collected until 
the completion of the experiments, and you will have to collect in per- 
son. 
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